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ABSTRACT 



The invention provides an identification data insertion and 
detection system for digital data which improves the detec- 
tion ratio when an image is converted into frequency com- 
ponents and electronic watermark data are embedded in a 
portion of an image having a strong frequency component 
and besides allows a statistical similarly to be calculated 
with a higher degree of accuracy. The identification data 
insertion system for digital data includes a DCT calculator 
for extracting blocks of 8x8 pixels from within a current 
image and performing DCT (discrete cosine transform) 
calculation for the blocks, a quantizer for quantizing data 
outputted from the DCT calculator, a quantization table for 
being referred to when the quantizer performs quantization, 
an electronic watermark data insertion element for inserting 
electronic watermark data into data after quantization, and 
an encoder for encoding data outputted from the electronic 
watermark data insertion element to produce MPEG data in 
which the electronic watermark data are inserted and out- 
putting the MPEG data. 

4 Claims, 8 Drawing Sheets 
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IDENTIFICATION DATA INSERTION AND pixel data such as a least significant bit (LSB) which has a 

DETECTION SYSTEM FOR DIGITAL DATA comparatively little influence on the picture quality. 

However, where this method is applied, electronic water- 

BACKGROUND OF THE INVENTION mar k data can be removed readily from an image. For 

1 Field of the Invention 5 exam P le > tf a low-pass filter is used, then information of the 

_ . . - . , LSB of a pixel is lost. Further, electronic watermark data are 

This invention relates to a system for processing a digital . . , £ . tl "- uuuu - ™« 

, i i . * r u lost also by image compression processing smce the image 

image, and more particularly to a system for embedding • . r , *. j * . . 

. ? . . i j .j » ■ j * u compression processing reduces the data amount by remov- 

electronic watermark data as identification data havmg spe- . F r . . 6 .... i 

... - . , , . • , | . 6 r mg mformation at such a portion which has a comparatively 

cial information into a digital image. _ ,.,7, . « , L * v. j i_ 

f , 10 little influence on the picture quality as mentioned above. 

2. Description of the Related Art Consequently, it is difficult to re-detect the electronic water- 
In recent years, illegal duplication of a digital image has mark data, 
become a serious problem. Another method is disclosed in Japanese Patent Laid- 

In order to prevent illegal duplication, a system has been Op en Application No. Heisei 6-315131. 

proposed wherein digital image data are ciphered such that 15 In tne melhod of ^ document just mentioned, a region 

only a reproduction system havmg a legal decipherment key which can bc replaccd> upon reproduction, by a peripheral 

can reproduce the ciphered digital image data. However, region causiflg deterioration of an image is detected 

once the cipher is deciphered, later duplication cannot be ^ a of of succe ssive frames, and the 

prevented. j eve j 0 f me re gj on 0 f an object of conversion is converted to 

Thus, in order to prevent illegal use and duplication of a 20 embed particular information. In this method, upon 

digital image, a method has been proposed wherein elec- reproduction, the region in which identification data are 

tronic watermark data which are special information are embedded is specified using the signal missing portion and 

embedded into the digital image itself. the replacement information, and the portion is corrected to 

As such electronic watermark data for a digital image as restore the image, 

described above, two kinds of electronic watermark data of 25 a further method is disclosed in lapanese Patent Laid- 

visible electronic watermark data and invisible electronic open Application No. Heisei 5-30466 wherein a video 

watermark data are available. signal is frequency converted and information having a 

The visible electronic watermark data are used such that frequency signal lower than a frequency band of the video 

a special character or symbol or the like is combined with an signal after the frequency conversion is embedded into the 

image so that it may be visually recognized. While the 30 video signal. In this method, the original video signal is 

visible electronic watermark data deteriorate the picture extracted using a high pass filter while embedded ideotifi- 

quality, they have an effect of visually appealins to a user of cation data are extracted using a low pass filter, 

a digital image for prevention of illegal use of the digital As another method which involves frequency conversion 

image. 35 of an image, a method has been proposed wherein an image 

An example of embedding of visible electronic watermark is frequency converted and electronic watermark data are 

data is disclosed in Japanese Patent Laid-Open Application embedded into a region of the video signal after the fre- 

No. Heisei 8-241403. According to the method disclosed in quency conversion in which strong frequency components 

the document just mentioned, visible electronic watermark are included (Nikkei Electronics. Apr. 22, 1996, No. 660, 

data are combined with an original image such that only the 4Q p. 13). 

brightness components of those pixels which correspond to i n this method, since electronic watermark data are 
an opaque portion of the electronic watermark data are embedded into frequency components, the electronic water- 
varied while color components of the pixels are not varied. mar k data are not lost even by image processing such as 
In this instance, the scaling value by which the brightness compression processing or filtering. Further, random num- 
components of pixels are varied is determined based on 45 D ers which exhibit a normal distribution are adopted as the 
color components, random numbers, values of pixels of the electronic watermark data to prevent interference between 
electronic watermark data and so forth. electronic watermark data and make it difficult to destroy 
On the other hand, the invisible electronic watermark data electronic watermark data without having a significant influ- 
are embedded into an image so that the picture quality may ence on the entire image, 

not be deteriorated. Since the invisible electronic watermark 50 According to the electronic watermark data embedding 

data little deteriorate the picture quality, they are character- method in the method described above, an original image 

ized in that they cannot be recognized visually. 501 is converted into frequency components using a DCT 

However, if special information which allows identifica- (discrete cosine transform) transformer 502 as seen in FIG. 
tion of an author is embedded as the electronic watermark 5. Then, n data which exhibit comparatively high values in 
data, then also after illegal duplication is performed, the 55 a frequency region are selected and individually represented 
author can be specified by detecting the electronic water- by f(l), f(2), . . . , f(n), and electronic watermark data 503 
mark data. On the other hand, if duplication inhibition (w(l), w(2), . . . , w(n)) are selected from within a normal 
information is embedded, then, for example, when a repro- distribution whose average is 0 and whose dispersion is 1. 
duction apparatus detects the duplication inhibition Then, F(i)=f(i)+ax|f(i)|xw(i) is calculated for each i by an 
information, it is possible to notify the user that duplication 60 electronic watermark data insertion element 504. Here, a is 
of the data is inhibited or to render a duplication prevention a scaling factor. Finally, DCT transform coefficients includ- 
mechanism in the reproduction apparatus operative to limit ing those frequency components in which f(i) are replaced 
duplication of the data by a video tape recorder (VTR) or the by F(i) are inverse transformed by an inverse DCT trans- 
like, former 509 to obtain an image 510 in which the electronic 

One available methods of embedding invisible electronic 65 watermark data is embedded, 

watermark data into a digital image is to embed special Electronic watermark data is detected using the following 

information as electronic watermark data into a portion of method. In the detection method, an original image and 
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electronic watermark data candidates w(i) (where i=l, w(i) and WF(i) is calculated in accordance with OWFxw/ 

2, . . , , n) must be known. (WFDxwD) making use of an inner product of vectors. 

Referring to FIG. 6, an original image 601 and an elec- Here, W=(WF(1), WF(2), . . . , WF(n)), w=(w(l), w(2), 

tronic watermark data containing image 602 are converted w (n)), WFD=absolute value of the vector WF, and 

into frequency components using DCT transformers 603 and s wD=absolute value of the vector w. When the statistical 

604, respectively. In a frequency region, values of factors similarity C has a value higher than a particular value, it is 

corresponding to f(l), f(2), . . . , f(n) in which electronic discriminated that the electronic watermark data are embed- 

watermark data are embedded are represented by F(l), ded. 

F(2), . . . , F(n), respectively. An electronic watermark data FIG. 7 shows a conventional system which embeds elec- 

extractor 605 extracts electronic watermark data W(i) based 10 tronic watermark data using such a method as described 

on f(i) and F(i) by calculation of W(i)=<F(i)-f(i))/f(i). Then, above, and FIG. 8 shows a conventional system which 

an inner product calculator 608 calculates, based on elec- detects electronic watermark data embedded by such a 

tronic watermark data 609, a statistical similarity between system as shown in FIG. 7. 

w(i) and W(i) in accordance with C-Wxw/(WDxwD) using Referring first to FIG. 7, in the system shown, a current 
inner products of vectors. Here, W<W(1), W(2), . . . , 15 imB&& 7Q1 is ^ idc6 into blocks 702 0 f g x 8 pixels and 
W(n)), w-(w(l), w(2), . . . , w(n)), WD-absohite value of the discrete cosine transformed by a DCT transformer 703, and 
vector W, and wD«absolute value of the vector w. A statis- mcn electronic watermark data 704 arc inserted into an 
tical similarity discriminator 610 discriminates, when the output of the DCT transformer 703 by an electronic water- 
statistical similarity C is higher than a particular value, that mark data insertion element 705. In the electronic watermark 
the pertaining electronic watermark data are embedded. 20 daU msert i on element 705, partial averages mentioned 

If electronic watermark data are embedded in an image above are calculated by a partial average calculator 706, and 

using the method described above, then they can be effec- a result of the calculation and the electronic watermark data 

tively used when an author who owns the original image 704 are multiplied by a multiplier 708. Then, an output of 

performs detection processing for digital image data which the multiplier 708 is multiplied by a constant (scaling factor) 

arc supposed to be an illegal duplicate. 25 a 707 by another multiplier 709. Then, a result of the 

However, since the method described above requires an multiplication is added to the image data from the DCT 

original image, although it is effective when an author who transformer 703 by an adder 710 to make image data in 

owns an original image performs detection processing for which the electronic watermark data are embedded. The 

image data which are supposed to be an illegal duplicate, a image data are quantized by a quantizer 711 with reference 

reproduction apparatus of a terminal cannot perform elec- to a quantization table 712 and then encoded by an encoder 

tronic watermark data detection processing because it does 713 to produce MPEG image data 714 in which the elec- 

not have the original image. Thus, another method which is tronic watermark data are embedded, and the MPEG image 

an improvement over the method described above and is data 714 are outputted. 

directed to terminal processing and particularly to an MPEG ^ Referring now to FIG. 8, in the system shown, MPEG 

(Moving Picture Experts Group) system has been proposed. image data 801 in which electronic watermark data are 

According to the improved method, an original image is embedded are decoded by a decoder 802 and then dequan- 

divided into blocks of 8 pixelsx8 pixels, and embedding and tized by a dequantizer 803, whereafter they are inverse DCT 

extraction processing of electronic watermark data is per- transformed by an inverse DCT calculator 804 to restore 

formed regarding each block as a processing unit. ^ original image data 805. Meanwhile, electronic watermark 

In electronic watermark data embedding processing, data data are extracted from the output of the dequantizer 803 by 

are first set successively as f(l), f(2), . . . , f(n) in the an electronic watermark data extractor 806. In the electronic 

ascending order of the frequency component of an AC watermark data extractor 806, after partial average calcula- 

component in a frequency region after discrete cosine trans- lion by a partial average calculator 807 and division by a 

form in MPEG coding processing is completed, and elec- 45 divider 808 are performed, sum totals WF(i) for one screen 

tronic watermark data w(l),w(2),...,2(n) are selected from are performed for each i by an adder 809. Then, inner 

within a normal distribution whose average is 0 and whose products of the sum totals W(i) and electronic watermark 

dispersion is 1, and then F(i)=f(i)+axavg(f(i))xw(i) is per- data 811 of an object of detection are calculated by an inner 

formed for each i. Here, a is a scaling factor, and avg(f(i)) product calculator 810 to obtain a statistical similarity 812. 

is a partial average which is an average of absolute values of 50 Tb en > if the statistical similarity 812 has a value higher than 

three neighboring points to f(i). Then, succeeding processing a particular value, then it is discriminated that the electronic 

of MPEG coding processing is performed replacing f(i) with watermark data are embedded in the MPEG image data 801. 

F(i). In the method disclosed in Japanese Patent Laid-Open 

Detection of electronic watermark data is performed in Application No. Heisei 6-315131, since electronic water- 
accordance with the following method. In this detection 55 mark information is not embedded in all frames, illegal 
method, an original image is not required, but only it is duplication cannot be prevented with regard to any frame in 
required that data candidates w(i) (where i=l, 2, . . . , n) be which the electronic watermark information is not embed- 
known. ded. Further, since successive frames represent still pictures 

In a frequency region of each block after dequantization afl d it is presupposed that there is no variation between 

of MPEG decompression processing is completed, data are 60 successive frames, where moving pictures which involve 

represented as F(l), F(2), . . . , F(n) in the ascending order strong movements are handled, a region into which elec- 

of the frequency component. An average value of absolute tronic watermark data are to be embedded cannot be speci- 

values of three neighboring points to F(i) is represented as fed. Consequently, electronic watermark cannot be embed- 

a partial average avg(F(i)), and electronic watermark data ded. 

W(i) is calculated in accordance with W(i)-F(i)/avg(F(i)). 65 Meanwhile, in the method disclosed in Japanese Patent 

Further, a sum total WF(i) of W(i) for one image is calcu- Laid-Open No. Heisei 5-30466, since electronic watermark 

lated for each i. Thereafter, a statistical similarity between data are embedded into a portion of an image which is lower 
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than a frequency region after frequency conversion of the factor of the data outputted from the quantizer as a partial 

image, the electronic watermark data can be removed readily average, a first multiplier for multiplying the electronic 

using a band-pass filter. watermark data by an output of the partial average calculator 

Further, with the method wherein electronic watermark for lhe ^dividual factors, a second multiplier for multiply- 

data are embedded into a portion having a strong frequency S mg ca <* factor of an output of the first multiplier by a 

component after frequency conversion, while electronic constant ™* !° var ? ^ ° f ^tronic 

watermark data cannot be removed using a filter or the like, watermark data to be inserted, and an adder for adding an 

. t . , # . ^ , - r. „ . output of the second multiplier and the output of the quan- 

electronic watermark data inserted after DCT processing are .. r f . . c f r ^ 

_ r . . , & uzer for the mdividual factors, 

sometimes lost by quantization processing. This is because, A . - 

. i r r . r» nn- p ii m According to another aspect of the present invention, 

where a value of any frequency component after DCT falls 10 & r \ * 

n • - * ^. . there is provided an identification data detection system for 

within a fixed range, it is converted into a particular repre- da(a ^ [Q M e]ecaonic watennark data 

sentative value in quantization processing. • _* jl *u -j »•« *• -j * * * r j- i 

n r "fis inserted by the identification data insertion system for digital 

Accordingly, due to an action of quantization, the values data described above and comprising a decoder for decoding 

of a partial average avg(f(i)) of three neighboring points compressed image data, a dequantizer for dequantizing data 

upon insertion and a partial average avg(F(i)) of the three 15 outputted tmm me decode r, an inverse DCT calculator for 

neighboring points upon detection are sometimes different performing inverse discrete cosine transform calculation for 

by a large amount. In such an instance, the statistical data oulputted from lhe deqU antizer and outputting image 

similarity between extracted electronic watermark data and data obtaincd by thc iaversc discrctc cosinc traQS f 0 r m 

electronic watermark data to be detected exhibits a low calculation, an electronic watermark data extractor for 

value and makes a factor which causes an erroneous dis- 20 cxtracting md outputting, from among the data outputted 

crimination result. from me decoder, those data which are supposed to be 

SUMMARY OF THE INVENTION electronic watermark data, an adder for adding data in units 

of a block of 8x8 pixels outputted from the electronic 

It is an object of the present invention to provide an watermark data extractor for individual factors over one 

identification data insertion and detection system for digital 25 screen, and an inner product calculator for calculating inner 

data which improves the detection ratio when an image is products of electronic watermark data to be detected and the 

converted into frequency components and electronic water- data outputted from the adder to calculate a statistical 

mark data are embedded in a portion of an image having a similarity and outputting the statistical similarity, 

strong frequency component. . 30 The electronic watermark data extractor may include a 

It is another object of the present invention to provide an partial average calculator for calculating an average of 

identification data insertion and detection system for digital absolute values of three neighboring points of each of data 

data which allows a statistical similarly to be calculated with m umts 0 f a matrix block of 8x8 pixels outputted from the 

a higher degree of accuracy. decoder as a partial average, and a divider for dividing the 

In order to attain the objects described above, according 35 data outputted from the decoder by the partial average 
to the present invention, taking it into consideration that outputted from the partial average calculator 
DCT coefficients are rounded by quantization, insertion of With the identification data insertion system and the 
electronic watermark data is performed not in a stage identification data detection system for digital data, since 
between DCT processing and quantization processing, but electronic watermark data insertion processing is performed 
after quantization. On the other hand, upon detection, elec- 4Q after quantization whereas electronic watermark data detec- 
tronic watermark data detection processing is performed tion processing is performed prior to dequantization, other- 
prior to dequantization. wise possible loss or disappearance of electronic watermark 

In the meantime, where insertion of electronic watermark data by quantization can be prevented, and the difference 

data is performed in a stage between DCT processing and between a partial average used upon insertion and another 

quantization processing similarly as in the conventional 45 partial average used upon detection is reduced, 

methods, when a partial average is to be calculated, an Consequently, a statistical similarity can be calculated with 

absolute value of a value obtained by multiplying an integral a higher degree of accuracy. Accordingly, the accuracy in 

value obtained by division of a corresponding value of a detection of electronic watermark data is improved, 

quantization further by the corresponding value of the [t is to be noted that the identification data insertion 

quantization table. 50 system and the identification data detection system for 

More particularly, according to an aspect of the present digital data may be combined so as to form a single 

invention, there is provided an identification data insertion identification data insertion and detection system for digital 

system for digital data for inserting electronic watermark data. 

data as identification data having special information into a According to a further aspect of the present invention, 

digital image, comprising a DCT calculator for extracting 55 there is provided an identification data insertion system for 

matrix blocks of 8x8 pixels from a current image and digital data, comprising a DCT calculator for extracting 

performing discrete cosine transform calculation for the matrix blocks of 8x8 pixels from a current image and 

blocks, a quantizer for quantizing data outputted from the performing discrete cosine transform calculation for the 

DCT calculator, a quantization table for being referred to blocks, an electronic watermark data insertion element for 

when the quantizer performs quantization, an electronic 60 inserting electronic watermark data into data after the dis- 

watermark data insertion element for inserting electronic crete cosine transform outputted from the DCT calculator, a 

watermark data into the data after the quantization, and an quantizer for quantizing data outputted from the electronic 

encoder for encoding data outputted from the electronic watermark data insertion element, a quantization table for 

watermark data insertion element. being referred to when the quantizer performs quantization 

The electronic watermark data insertion element may 65 and when the electronic watermark data insertion element 

include a partial average calculator for calculating an aver- calculates a partial average, and an encoder for encoding 

age of absolute values of three neighboring points of each data outputted from the quantizer. 
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The electronic watermark data insertion element may lowing description and the appended claims, taken in con- 

inchide a partial average calculator for calculating, using the junction with the accompanying drawings in which like 

quantization table when a partial average of the data out- parts or elements are denoted by like reference characters, 
putted from the DCT calculator is to be calculated, an 

average among three neighboring points of absolute values 5 BRIEF DESCRIPTION OF THE DRAWINGS 

of values ; obtained each by multiplying an integer value FIG. 1 is a block diagram of an electronic watermark data 

obtained by dividing ; a value of each factor outputted from ^on g (£) which ^ ^ , ie(J 

the DCT calculator by a corresponding value of the quan- _„ «..,.,. , , 

tization table by the corresponding value of the quantization J ™ G : 2 15 a block of an electronic watermark data 

table as a partial average of the pertaining factor, a first 10 Election system to winch the present invention is applied; 

multiplier for multiplying an output of the partial average FIG- 3 is a block diagram of another electronic watermark 

calculator by the electronic watermark data for the indi- data insertion system to which the present invention is 

vidual factors, a second multiplier for multiplying data applied; 

outputted from the first multiplier by a constant to be used FIG, 4 is a block diagram of another electronic watermark 

to vary the size of the electronic watermark data to be 15 data detection system to which the present invention is 

embedded, and an adder for adding the data outputted from applied; 

the DCT calculator and data outputted from the second pjc. 5 is a block diagram showing a conventional elec- 

multiplier for the individual factors. tron jc wa termark data insertion system; 

According to a still further aspect of the present invention, pj G g is a block diagram showing a conventional elec- 
tee is provided an identification data detection system for 20 tromc watermark data detection system; 
digital data adapted to detect electronic watermark data no. 7 is a block diagram showing another conventional 
inserted by the identification data insertion system for digital eleclr0Qic watermark data insertion ^ 
data described above and comprising a decoder for decoding 

compressed image data, a dequantizer for dequantizing data FIG. 8 is a block diagram showing another conventional 

outputted from the decoder, an inverse DCT calculator for * electronic watermark data detection system, 

performing . inverse discrete cosine transform calculation for DESCRIPTION OF THE PREFERRED 

data outputted from the dequantizer and outputting image EMBODIMENTS 
data obtained by the inverse discrete cosine transform 

calculation, an electronic watermark data extractor for Referring first to FIG. 1, there is shown an electronic 

extracting and outputting, from among the data outputted 30 watermark data insertion system to which the present inven- 

from the dequantizer, those data which are supposed to be tion is applied. The electronic watermark data insertion 

electronic watermark data, an adder for adding data in units system shown includes a DCT calculator 103 for extracting 

of a block of 8x8 pixels outputted from the electronic blocks 102 of 8x8 pixels from within a current image 101 

watermark data extractor for individual factors over one which makes an object of insertion of electronic watermark 

screen, and an inner product calculator for calculating inner 35 data and performing DCT (discrete cosine transform) cal- 

products of electronic watermark data to be detected and the culation for the blocks 102, a quantizer 104 for quantizing 

data outputted from the adder to calculate a statistical data outputted from the DCT calculator 103, a quantization 

similarity and outputting the statistical similarity. table 105 for being referred to when the quantizer 104 

The electronic watermark data extractor may include a performs quantization, an electronic watermark data inser* 

partial average calculator for calculating an average of 40 tion element 107 for inserting electronic watermark data 106 

absolute values of three neighboring points of each of data into data after quantization, and an encoder 113 for encoding 

in units of a block of 8x8 pixels outputted from the dequan- data outputted from the electronic watermark data insertion 

tizer as a partial average, and a divider for dividing the data element 107 to produce MPEG data 114 in which the 

outputted from the dequantizer by the partial average out- electronic watermark data 106 are inserted and outputting 

putted from the partial average calculator. 45 toe MPEG data 114. 

With the identification data insertion system wherein The electronic watermark data insertion element 107 

insertion of electronic watermark data is performed in a includes a partial average calculator 108 for calculating a 

stage between DCT processing and quantization processing partial average of three neighboring points of data outputted 

similarly as in the conventional methods and with the 5Q from the quantizer 104 in accordance with avg(f(i))=(|f(i- 

identification data detection system, when a partial average l)|+|f(i)|+|f(i+l)|)/3, a multiplier 110 for multiplying outputs 

is to be calculated, an absolute value of a value obtained by of the electronic watermark data 106 and the partial average 

multiplying an integral value obtained by division of a calculator 108 for individual factors, another multiplier 111 

corresponding value of a quantization table further by the for multiplying each factor of an output of the multiplier 110 

corresponding value of the quantization table. Consequently, 5S by a constant 109 which is used to vary the size of the 

the difference between a partial average used upon insertion electronic watermark data to be inserted, and an adder 112 

and another partial average used upon detection is sup- for adding an output of the multiplier 111 and the output of 

pressed and a statistical similarity can be calculated with a toe quantizer 104 for individual factors, 

higher degree of accuracy. Accordingly, the accuracy in The electronic watermark data insertion element 107 

detection of electronic watermark data is improved. 6Q performs calculation defined by the following expression: 

It is to be noted that the identification data insertion 

system and the identification data detection system for F{iHVh™&mMn^** 

digital data may be combined so as to form a single where f(i) is the data outputted from the quantizer 104, 

identification data insertion and detection system for digital avg(f(i)) is the data outputted from the partial average 

d ata * 65 calculator 108, w(i) is the electronic watermark data 106, the 

The above and other objects, features and advantages of constant is the constant 109, and i is the number of each 

the present invention will become apparent from the fol- factor after zigzag scanning of a block of 8x8 pixels. 
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Referring now to FIG. 2, there is shown an electronic neighboring factors, that is, (|f(i-l)|+|f(i)|+|£(i+l)D/3, and 

watermark data detection system to which the present inven- F(i) is the new frequency component in which the electronic 

tion is applied. The electronic watermark data detection watermark data 106 are inserted. 

system detects electronic watermark data inserted in an The encoder 113 encodes the data F(i) outputted from the 

image by the electronic watermark data insertion system of s electronic watermark data insertion element 107 by MPEG 

FIG. 1 described above. The electronic watermark data processing to produce MPEG data 114 in which the elec- 

detection system includes a decoder 202 for decoding tronic watermark data 106 are embedded. 

MPEG data 201 which make an object of detection of Now, operation for detection of electronic watermark data 

electronic watermark data, a dequantizer 203 for dequantiz- is described. 

ing data outputted from the decoder 202, an inverse DCT 10 First, MPEG data 201 which make an object of detection 

calculator 204 for performing inverse DCT calculation for is subject to MPEG decoding by the decoder 202. The data 

data outputted from the dequantizer 203 to obtain image data decoded by the decoder 202 are dequantized by ordinary 

205 and outputting the image data 205, an electronic water- MPEG decoding processing by the dequantizer 203 and 

mark data extractor 206 for extracting, from among data inverse DCT calculated by the inverse DCT calculator 204 

outputted from the decoder 202, those data which are 15 to produce reproduction image data 205. 

supposed to be electronic watermark data, an adder 209 for The electronic watermark data extractor 206 extracts the 

cumulatively adding data in units of a matrix block of 8x8 data decoded by the decoder 202, performs calculation of 

pixels outputted from the electronic watermark data extrac- F(i)/avg(F(i)) and extracts those data which are supposed to 

tor 206 for individual factors over one screen, and an inner be embedded electronic watermark data. Here, F(i) is the 

product calculator 210 for calculating inner products of 20 frequency component of a decoded block of 8x8 pixels, and 

electronic watermark data 211 to be detected and data avg(F(i)) is the partial average ((|F(i-l)|+|F(i)|+|F(i+l)|/3. 

outputted from the adder 209 to obtain a statistical similarity The adder 209 calculates a sum total of the extracted data in 

212 and outputting the statistical similarity 212. units of 8x8 pixels outputted from the electronic watermark 

The electronic watermark data extractor 206 includes a data extractor 208 for individual factors over one screen, 

partial average calculator 207 for calculating a partial aver- 25 After the sum totals of the extracted data of the one screen 

age of three neighboring points of data in units of a block of are calculated, the inner product calculator 210 calculates 

8x8 pixels outputted from the decoder 202 in accordance inner products of the sum totals and electronic watermark 

with avg(F(i))-(|F(i-l)k|F(0l + l F (i +1 )l)/ 3 » and a divider 208 data 211 to be detected to produce a statistical similarity 212 

for dividing data F(i) outputted from the decoder 202 by a and outputs the statistical similarity 212. 

partial average avg(F(i)) outputted front the partial average 30 If the statistical similarity 212 is higher than a particular 

calculator 207. fixed value, then it is discriminated that electronic water- 

The electronic watermark data extractor 206 performs mark data equivalent to the electronic watermark data 211 

calculation defined by the following expression: are embedded in the MPEG data 201. 

In this instance, if the electronic watermark data 211 

w{i)=F(C)faYg(F(i)) 35 represent inhibition of duplication, then the reproduction 

u r?/-\ ■ *u j * * j a- 4U a j apparatus in which the electronic watermark data detection 

where Ffi) is the data outputted from the decoder 202, . • • . _. . 1 . 

/c/-w ■ *u ** 1 , j r *u *■ 1 system is incorporated can take such a countermeasure as to 

avg(F(i)) is the partial average outputted from the partial . \., , 5 * ,« . . A „«, . , 

1 1 . -inn «7/-\ • .l /* L- u j inhibit duplication of the image data 205 produced therein, 

average calculator 207, W(i) is the data which are supposed „ , " . .i_ 1 . ■ 

4 • 1 j 1 t • * 1 a ♦ * « a c/ *u Referring now to FIG. 3, there is shown another electronic 

to mclude electronic watermark data outputted from the t t j * • . 4 . , . , 4 , 

. L . - , , A j • * it- t_ 4 o watermark data insertion system to which the present inven- 

electronic watermark data extractor 206, and 1 is the number 4 . . t . , ™ . ' 4 . *, , . 

f , e a ri_iir 00 -i tion is applied. The electronic watermark data insertion 

of each factor ot a block oi 8x8 pixels. . . , , nr^r i i * mi e *■ 

T 4L f « c - r system includes a DCT calculator 303 for extracting matrix 

In the following, operation for insertion and detection of .* . r o~o ■ i c_ — 4iM 

, . . | j . n 4 l i i • t , Ai blocks 302 of 8x8 pixels from mthin a current image 301> 

electronic watermark data of the electronic watermark data ... . . * ■ : «j 

, . . . . ... -4U r which makes' an object of insertion of electronic watermark 

insertion and detection systems desenbed above with refer- „ \ t -, - ~ - - i^wn— i~ ~i X- - c 1 ™w 

* r-F^o * j « * u j ■ j * -i *u f 45 ^data and performing DCT calculation for the blocks 302, an 

ence to FIGS. 1 and 2 is described in detail with reference L , -~—r *~a v • 1 * i fle * ■ - 

to FIGS l and 2 electronic yatermarkjata insertion element 305 for insert- 

„. ' 4 - "r . ri^ - * ij, ing electronic" watermark data-304-int6Tdata-for-wbichtriei 

First, operation for msertion of electronic watermark data _ - ;t- — r — y , - . w-- 

. , \. r , DCT transform -has-been" performed, a .quantizer 3113 for 

is described ~ ■ ■ ' ~ ~ — ~- j " — ' ' ' 

A /. irti ■ t ac 1- ui 1 c o o quantizing ^data" outputted~from~the- electronic- watermark 

A current image 101 is extracted for each block of 8x8 *} t - r „ v ^r-^«*r^ - . ^ o-.-, r 1 

■ 1 u 1 * kinr o . 1 50 data-msertion element 305^ a quantization-table-312 for 

pixels by ordinary MPEG compression processing, and the ."s c - , A , ' --- %n - A ~ c - 

DCT calculator 103 performs DCT calculation for the thus be ]°Z refc ™ d J° quantizer 104 performs quanU- 

^ * j j * * r r zation and when the electronic watermark data insertion 

extracted data. Data of frequency components obtained by t . - A _ , , t ^ , , , 

.u * ■* \ , f + M j element 305 calculates a partial average, and an encoder 313 

me DCT calculation of the DCT calculator 103 are quantized - _ - . , _ * , - -- ,n j 

f t 4 , 4 . L1 injfl _ for_encoding"data outputted-from-the quantizer-311, and 

with reference to the quantization table 105 by the quantizer „ . .^x/dep a 7 Ti >< • "t.- u \u ~i • K 

- Ajl , . . ^ . | j . ,. ; , , 1fl - 5S outputtmg-MPEG-data-314-m-which-the-electromc-water- 

104. The electronic watermark data insertion element 107 maAdata-304 are embedded ^ ^ - 1 

embeds electronic watermark data 106 into the data output- * ™. . ~. ■ , ' 4 . . - n _ 

A , c ■% t\A t * t *i_ i * • The electromc watermark data insertion element 305 

ted from the quantizer 104. In this instance, the electromc , . ^. , . . A r . . i . 

i j 7 • i mm jL • mcludes a partial average calculator 307 for calculating a 

watermark data msertion element 107 performs processing . , r - , . & t „ , - . . . 

, 4 , i i c lL c ii • ■ partial average of data outputted from the DCT calculator 

equivalent to calculation of the following expression: _ $ n - ... r & 4 t . r 4 . ^ . . . u . 

M & r 60 303 with reference to the quantization table 3 12, a multiplier 

/•(O^O+avg^xvK^constant 308 for multiplying the output of the partial average calcu- 
lator 307 by the electronic watermark data 304 for individual 

where i is the number of a factor of a frequency component factors, another multiplier 309 for multiplying the data 

of the block of 8x8 pixels, w(i) is the value of each factor outputted from the multiplier 308 by a constant a 306 which 

of the electronic watermark data 106, the constant is the 65 is used to vary the size of the electronic watermark data to 

constant 109, f(i) is each of the factors of the 8x8 pixel be inserted, and an adder 310 for adding the data outputted 

block, avg(f(i)) is the average of absolute values of three from the DCT calculator 303 and the data outputted from the 
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multiplier 309 for individual factors. The electronic water- 
mark data insertion element 305 thus performs processing 
equivalent to calculation of the following expression: 

F(f)=^i)+avg(^i))xH<i>coiistant s 

where f(i) is the data outputted from the DCT calculator 303, 
avg(f(i)) is the data outputted from the partial average 
calculator 307, w(i) is the electronic watermark data 304, the 
constant is the constant a 306, and i is the number of a factor 
after the data outputted from the DCT calculator 303 are 10 
zigzag scanned. 

Here, the partial average calculator 307 in the electronic 
watermark data insertion element 305 performs, in order to 
calculate a partial average avg(f(i)) of three neighboring 
points of the data outputted from the DCT calculator 303, 15 
processing equivalent to calculation of the following expres- 
sions: 

V(i-l)«|itttegral part of (ft*-l)/0(i-l))xeO - -l)| 

KOHintegral part of (/(O/GCOW)! 20 

V(i+lHintegral part of (/(i+l)/G(i+l))x(?(f+l)| 

Referring now to FIG. 4, there is shown another electronic 25 
watermark data detection system to which the present inven- 
tion is applied. The electronic watermark data detection 
system detects electronic watermark data inserted by the 
electronic watermark data insertion system described above 
with reference to FIG. 3. To this end, it includes a decoder 30 
402 for decoding MPEG data 401 which make an object of 
detection of electronic watermark data, a dequantizer 403 for 
dequantizing data outputted from the decoder 402, an 
inverse DCT calculator 404 for performing inverse DCT 
calculation for data outputted from the dequantizer 403 to 35 
obtain image data 405 and outputting the image data 405, an 
electronic watermark data extractor 406 for extracting, from 
among the data outputted from the dequantizer 403, those 
data which are supposed to be electronic watermark data and 
outputting the extracted data, an adder 409 for adding data 40 
in matrix units of an 8x8 pixel block outputted from the 
electronic watermark data extractor 406 for individual fac- 
tors over one screen, and an inner product calculator 410 for 
calculating inner products of data outputted from the adder 
409 and electronic watermark data 411 to be detected to 45 
obtain a statistical similarity 412 and outputting the statis- 
tical similarity 412. 

The electronic watermark data extractor 406 includes a 
partial average calculator 407 for calculating a partial aver- 
age of three neighboring points of data in units of 8x8 pixel 50 
block outputted from the dequantizer 403 in accordance with 
avg(F(i))=(|F(i-l)|+|F(i)|+|F(i+l)|)/3, and a divider 408 for 
dividing the data F(i) outputted from the dequantizer 403 by 
the partial average avg(F(i)) outputted from the partial 
average calculator 407. Here, F(i) is the frequency compo- 55 
nent of an 8x8 pixel block outputted from the dequantizer 
403, and i is the number of each factor after zigzag scanning 
of the 8x8 pixel block. 

The electronic watermark data extractor 406 performs 
calculation in accordance with the following expression: eo 

where F(i) is the data outputted from the inverse DCT 
calculator 404, avg(F(i)) is the partial average outputted 
from the partial average calculator 407, and W(i) is the data 65 
outputted from the electronic watermark data extractor 406 
and supposed to include electronic watermark data. 
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In the following, operation for insertion and detection of 
electronic watermark data of the electronic watermark data 
insertion and detection systems described above with refer- 
ence to FIGS. 3 and 4 is described in detail with reference 
to FIGS. 3 and 4. 

First, operation for insertion of electronic watermark data 
is described. 

A current image 301 is extracted for individual blocks 302 
of 8x8 pixels by ordinary MPEG compression processing, 
and the DCT calculator 303 performs DCT calculation for 
the thus extracted data. The electronic watermark data 
insertion element 305 embeds electronic watermark data 304 
into the data of frequency components obtained by the DCT 
transform. In this instance, the electronic watermark data 
insertion element 305 performs processing equivalent to 
calculation of the following expression: 

^ r (0 =, A0 +av gC/(0) XH '(0 xconstailt 

where i is the factor number after zigzag scanning of a block 
of 8x8 pixels, f(i) is the value of each coefficient obtained by 
the DCT transform, w(i) is the value of each factor of the 
electronic watermark data 304, and avg(f(i)) is the average 
of absolute values of three neighboring points of each factor. 
Here, avg(f(i)) is calculated in accordance with the follow- 
ing expressions: 

V(t-l)Hintegral part of (rtt-l)/2O"-l))x0(M)| 
K0=l"itegral part of (WQ(i))*Q($ 
V(M-lHintegral part of (f{i+l)IQ{M))xQ(H-l)\ 
avg(/(0)-W-l)+^>V('+l))/3 

The quantizer 311 refers to the quantization table 312 to 
perform quantization processing for the data outputted from 
the electronic watermark data insertion element 305, and the 
encoder 313 performs encoding of the data outputted from 
the quantizer 311 to produce MPEG data 314. 

Now, operation for detection of electronic watermark data 
is described. 

First, MPEG data 401 which make an object of detection 
are subject to MPEG decoding processing by the decoder 

402. The data decoded by the decoder 402 are dequantized 
by ordinary MPEG decoding processing by the dequantizer 

403, and inverse DCT calculated by the inverse DCT cal- 
culator 404 to produce image data 405. 

The electronic watermark data extractor 406 extracts 
decoded data from the dequantizer 403, performs calculation 
of F(i)/avg(F(i)) and extracts data supposed to be embedded 
electronic watermark data. Here, F(i) is the value of each 
factor after zigzag scanning in units of an 8x8 pixel block 
outputted from the dequantizer 403, and avg(F(i)) is the 
partial average flF(i-l)|+|F(i)|+|F<(i+l)|)/3. The adder 409 
calculates sum totals of the extracted data in units of an 8x8 
pixel block outputted from the electronic watermark data 
extractor 406 for individual factors over one screen. After 
the sum totals of the extracted data for one screen are 
calculated, the inner product calculator 410 calculates inner 
products of the sum totals and the electronic watermark data 
411 to be detected to obtain a statistical similarity 412 and 
outputs the statistical similarity 412. 

If the statistical similarity 412 is higher than a particular 
fixed value, then it is discriminated that electronic water- 
mark data equivalent to the electronic watermark data 411 
are embedded in the MPEG data 401. 

In this instance, if the electronic watermark data 411 
represent inhibition of duplication, then the reproduction 
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apparatus in which the electronic watermark data detection 
system is incorporated can take such a countermeasiire as to 
inhibit duplication of the image data 405 produced therein. 

Having now fully described the invention, it will be 
apparent to one of ordinary skill in the art that many changes 
and modifications can be made thereto without departing 
from the spirit and scope of the invention as set forth herein. 

What is claimed is: 

1. An electronic watermark insertion apparatus, compris- 
ing: 

conversion means for performing discrete cosine trans- 
form for an original image to convert the original image 
into data f(i) of frequency components, i being 1, 
2, . . . , n; 

electronic watermark insertion means for multiplying a 15 
partial average of the data f(i) and electronic watermark 
data w(i) determined in advance for the individual 
factors (i), i being 1, 2, . . . , n, and adding the data f(i) 
to the results obtained by multiplying the results of the 
multiplication by a constant to determine an electronic 20 
watermark containing data F(i), i being 1, 2, . . . , n; 

quantization means for quantizing the data F(i) with a 
quantization step sizes Q(i), i being 1, 2, . . . , n; and 

coding means for compression coding the data after the 
quantization; 

the partial average of the data f(i) being given by (V(i- 
l)+V(i)+V(i+l))/3 where V(i-l)-|integral part of (f(i- 
l)/Q(i-l)xQ(i-l)|, V(0=|integral part of (f(i)/Q(i)xQ 
(i)|, and V(i+l)-|integral part of (f(i+l)/Q(i+l)x 
Q(i+l)l- 

2. An electronic watermark detection apparatus to which 
data obtained by performing a discrete cosine transform for 
an original image to convert the original image into data f(i) 
of frequency components, i being 1, 2, . . . , n, multiplying 
a partial average of the data f(i) and electronic watermark 
data w(i) determined in advance for the individual factors 
(i), i being 1, 2, . . . , n, and adding the data f(i) to the results 
obtained by multiplying the results of the multiplication by 
a constant to determine electronic watermark containing 
data F(i), i being 1, 2, . . . , n, quantizing the data F(i) with 
quantizaiton step sizes Q(i), i being 1, 2, . . . , n, and then 
compression coding the data after the quantization, the 
partial average of the data f(i) being given by 



25 



30 



35 



(Ki-i)+v(0+vo'+i))/3 

where V(i-l)=|integral part of (f(i-l)/Q(i-l)xQ(i-l)|, V(i)= 
|tntegral part of (f(i)/Q(i)xQ(i)|, and V(i+l)=|integral part of 
(fi+l)/Q(i+l)xQ(i+l)|, the apparatus comprising: 

decoding means for decoding the compression coded 
data; 

dequantization means for dequantizing data F(i) obtained 

by the decoding, i being 1, 2, . . . , n; 
inverse discrete cosine transform means for performing an 

inverse discrete cosine transform for data obtained by 

the dequantization in order to restore the original 

image; and 

electronic watermark detection means for calculating a 
statistical similarity between values obtained by divid- 
ing the data F(i) by a partial average (|F(i-l)|+|F(i)|+ 



45 



50 



55 



60 



|F(i+l)|)/3 of the data F(i) and the electronic watermark 
data to discriminate whether or not the electronic 
watermark is included in the data supplied to said 
electronic watermark detection apparatus. 

3. An electronic watermark insertion method, comprising: 
a conversion step of performing a discrete cosine trans- 
form for an original image in order to convert the 
original image into data f(i) of frequency components, 
i being 1, 2, . . . , n; 

an electronic watermark insertion step of multiplying a 
partial average of the data f(i) and electronic watermark 
data w(i) determined in advance for the individual 
factors (i), i being 1, 2, . . . , n, and adding the data f(i) 
to the results obtained by multiplying the results of the 
multiplication by a constant to determine an electronic 
watermark containing data F(i), i being 1 , 2, . . . , n; 

a quantization step of quantizing the data F(i) with 
quantization step sizes Q(i), i being 1, 2, . . . , n; and 

a coding step of compression coding the data after the 
quantization; 

the partial average of the data f(i) being given by (V(i- 
l)+V(i)+V(i+l))/3 where V(i-l)«|integral part of (f(i- 
l)/Q(i-l)xQ(i-l)|, and V(i+l)=|integral part of (f(i+l)/ 
Q(i+l)xQ(i-l)|. 

4. An electronic watermark detection method to which 
data obtained by performing a discrete cosine transform for 
an original image to convert the original image into data f(i) 
of frequency components, i being 1, 2, . . . , n, multiplying 
a partial average of the data f(i) and electronic watermark 
data w(i) determined in advance for the individual factors 
(i), i being 1, 2, . . . , n, and adding the data f(i) to the results 
obtained by multiplying the results of the multiplication by 
a constant in order to determine an electronic watermark 
containing data F(i), i being 1, 2, . . . , n, quantizing the data 
F(i) with quantization step sizes Q(i), i being 1, 2, . . . , n, 
and then compression coding the data after the quantization, 
the partial average of the data f(i) being given by 

(V(i-l>V(i>V(i+l))/3 

where V(i-l)=|integral part of (fi(i-l)/Q(i)xQ(i-l)|, V(i)= 
integral part of (f(i)/Q(i)xQ(i)|, and V(i+l)=|integral part of 
(f(i+l)/Q(i+l)xQ(i+l)|, the method comprising: 

a decoding step of decoding the compression coded data; 
a dequantization step of dequantizing the data (i) obtained 

by the decoding, i being 1, 2, . . . , n; 
an inverse discrete cosine transform step of performing an 

inverse discrete cosine transform for data obtained by 

the dequantization in order to restore the original 

image; and 

an electronic watermark detection step of calculating a 
statistical similarity between values obtained by divid- 
ing the data F(i) by a partial average (|F(i-l)|+|F(i)|+ 
|F(i+l)[)/3 of the data F(i) and the electronic watermark 
data to discriminate whether or not the electronic 
watermark is included in the data supplied to the 
electronic watermark detection method. 
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